Skip to content

fix: skip unhealthy workers in proxy GET endpoints - #232

Open
Bevisy wants to merge 1 commit into
vllm-project:mainfrom
Bevisy:fix/proxy-get-healthy-worker
Open

Bevisy wants to merge 1 commit into
vllm-project:mainfrom
Bevisy:fix/proxy-get-healthy-worker

Conversation

@Bevisy

@Bevisy Bevisy commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Purpose

Fixes #135/v1/models (and other proxy GET endpoints) returns 500 when any worker is unhealthy.

select_first_worker returned the first registered worker without checking health state. worker_registry.get_all() returns all workers including unhealthy ones, so when the first worker was temporarily down, proxy_get_request proxied to a dead worker and returned 500. This affected /v1/models, /get_server_info, and /get_model_info — all of which route through proxy_get_requestselect_first_worker.

Root-cause fix in the shared helper: filter for a healthy worker (is_healthy()) before returning; fall back to the existing error path when none are healthy.

Test Plan

New unit tests in routers::http::router::tests:

  • test_select_first_worker_skips_unhealthy — marks worker1 unhealthy, asserts worker2 is selected
  • test_select_first_worker_all_unhealthy — marks all unhealthy, asserts error

Test Result

running 3 tests
test routers::http::router::tests::test_select_first_worker_regular ... ok
test routers::http::router::tests::test_select_first_worker_skips_unhealthy ... ok
test routers::http::router::tests::test_select_first_worker_all_unhealthy ... ok

test result: ok. 3 passed; 0 failed

Built with cargo test --lib routers::http::router::tests::test_select_first_worker (rust:latest container, aarch64).

select_first_worker returned the first registered worker without checking
its health state, so /v1/models and other proxy GETs returned 500 when the
first worker was temporarily down. Filter for a healthy worker instead;
fall back to the existing error when none are healthy.

Fixes vllm-project#135

Signed-off-by: Binbin Zhang <binbin36520@gmail.com>
@hsliuustc0106 hsliuustc0106 added the bug Something isn't working label Sep 14, 2026
@wuhang2014
wuhang2014 requested a balanced review from Copilot September 14, 2026 09:46

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Worker selection still bypasses open circuit breakers despite the proxy path requiring an available worker.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Updates proxy GET routing to avoid unhealthy workers.

Changes:

  • Selects a healthy worker for proxy GET endpoints.
  • Adds tests for partial and complete worker unavailability.
File summaries
File Description
src/routers/http/router.rs Filters worker selection by health and tests fallback behavior.
Review details
  • Files reviewed: 1/1 changed files
  • Comments generated: 2
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +344 to +346
match workers.into_iter().find(|w| w.is_healthy()) {
Some(worker) => Ok(worker.url().to_string()),
None => Err("No healthy workers are available".to_string()),
Comment on lines +341 to +342
// Prefer a healthy worker so proxy GETs (e.g. /v1/models) don't 500
// when the first registered worker is temporarily down.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

/v1/models returns 500 when any worker is unhealthy

3 participants